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DETAILED ACTION 



Claim Rejections - 35 USC §112 



1 . The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

2. Claims 5-7, 27-32, 34-40 are rejected under 35 U.S.C. 1 12, second paragraph, 
as being indefinite for failing to particularly point out and distinctly claim the subject 
matter which applicant regards as the invention. 

Regarding claims 5 - 7, 27 -32, the phrase " at least one each story segments" is 
not clear. Appropriate correction is required. The examiner interpreted this as "at least 
one of the story segments." 

Regarding claims 34 - 40, the phrase "the user interface of claim" does not 
particularly point out to a specific claim, making the claims indefinite. Appropriate 
correction is required. The examiner interpreted this as: 

Claims 34, 37, 39, 40 depend on claim 33; 

Claim 35 depends on claim 34. 
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Claim Rejections - 35 USC § 102 



3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the 
United States before the invention thereof by the applicant for patent, or on an international application 
by another who has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this 
title before the invention thereof by the applicant for patent. 

The changes made to 35 U.S.C. 102(e) by the American Inventors Protection Act 
of 1999 (AIPA) do not apply to the examination of this application as the application 
being examined was not (1) filed on or after November 29, 2000, or (2) voluntarily 
published under 35 U.S.C. 122(b). Therefore, this application is examined under 35 
U.S.C. 102(e) prior to the amendment by the AIPA (pre-AlPA 35 U.S.C. 102(e)). 

4. Claims 1 -10, 13-17, 27-34, 37, 39 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Dimitrova (US 6,363,380). 

Regarding claim 1, Dimitrova discloses a story segmentation device 10 receives 
multimedia signal input, the story segmentation device 10 comprising: video shot 
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parsing device 102, audio parsing device 104, text parsing device 106 and event model 
recognition device 112, wherein the combination of these devices read on the story 
segment identifier being claimed, and the classification device 1 14 reads on the 
classifier being claimed (see figures 1-2 and 6A). 

Regarding claim 2, Dimitrova discloses the multimedia signal includes closed 
caption, and the text parsing device 106 partitions the text stream into text segment 
corresponding to story segment and the classification device 1 14 associates the one or 
more classification to the at least one each story segment based on the at least one text 
segment (see figures 1-2 and 6A). 

Regarding claim 3, Dimitrova discloses the multimedia signal includes audio 
stream, and the audio parsing device 106 partitions the audio stream into audio 
segment corresponding to story segment, and the classification device 114 associates 
the one or more classification to the at least one each story segment based on the at 
least one audio segment (see figures 1-2 and 6A). 

Regarding claim 4, Dimitrova discloses a system as discussed in the rejection of 
claim 3. Dimitrova further discloses speech recognizer or voice recognition software and 
closed caption (see col. 1, line 61- col. 2, line 60). Inherently, the audio-to-text converter 
is included in the classification device for converting audio segment to text segment. 
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Regarding claim 5, Dimitrova discloses retrieve broadcast television signal 
including video shots based on their visual content (see col. 1, line 40- col. 2, line 17) or 
recognizing a person (see col. 12, line 1). Inherently, the system includes a visual 
characterizer for providing a visual characterization of at least one each story segment 
of the plurality of story segments based on an image content of the at least one each 
story segment, and the classifier associates the one or more classification to the at least 
one each story segment based on the visual characterization. 

Regarding claim 6, Dimitrova discloses the visual characterizer includes a 
drawing (e.g., flow charts) a face recognition (e.g., lip, eye, and head movement) (see 
col. 2, lines 1-10. Inherently, a figure recognizer is included in the system for 
recognizing the drawing or face based on the image content. 

Regarding claim 7, Dimitrova discloses the visual characterizer includes a flesh 
tone recognizer (see col. 12, lines 1-14). 

Regarding claim 8, Dimitrova discloses the "story segment identifier" partitions 
the multimedia signal based on a video cut (see col. 9, lines 52-65). 

Regarding claim 9, Dimitrova discloses the one or more classification include 
keyword (see figure 6A). 
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Regarding claim 10, Dimitrova discloses a system as discussed in the rejection 
of claim 1. Dimitrova further discloses one or more key frame in each story segment 
(see col. 10, lines 20-26). Inherently, each story segment of the plurality of story 
segments includes one or more scenes and one or more key frames correspond to a 
frame within each or the one or more scene. 

Regarding claim 13, Dimitrova discloses the multimedia signal is communicated 
from at least digital signal broadcast, analog signal broadcast, internet connection (see 
col. 8, lines 45-55). 

Regarding claim 14, Dimitrova discloses the story segmentation device further 
including a storage device that stores the plurality of story segments (see figure 1 or col. 
8, lines 46-48). 

Regarding claim 15, Dimitrova discloses the storage device is a VCR (see col. 8, 
lines 50-51). 

Regarding claim 16, Dimitrova discloses at least one of the one or more key 
frames is a video clip (see col. 15, lines 52-56). 

Regarding claim 17, Dimitrova discloses a story segment retrieval device 
including comparing the input key frame or key word with the key frame or key word 
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stored in the storage device and retrieving only one or more story segments that 
matches the input key frame or key word, and the corresponding index will specify the 
video story with the key frame with is most similar to the image input and the output will 
be displayed on the display device 30 (see col. 14, line 54 - col. 5, line 36 and figure 1). 
Inherently, a "filter" is included in the system to identify one or more filtered story 
segments of the plurality of story segments based on the one or more classifications 
that are associated with each story segment. 

Regarding claim 27, Dimitrova discloses a video device comprising: 
a classification device that classifies a plurality of segments of a multimedia signal by 
producing a classification based on at least one of text, audio, or visual information 
associated with each segment of the plurality of segments, and 
a retrieval device that facilitate a selection of an at least one segment of the plurality of 
segments by matching the classification of the at least one of plurality of segments to 
and at least one user preference (see figure 1 and col. 14, line 54-col. 16, line 7). 
Inherently, the key frame is displayed on the display device 30. 

Regarding claim 28, Dimitrova discloses storage device 20 coupled to the display 
device 30 (see figure 1). Inherently, the storage device 20 communicates the at least 
one of each segment of the multimedia signal to the display device 30 based on the 
selection of the segment. 
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Regarding claim 29, Dimitrova discloses the storage device 20 that stores the 
plurality of segments (see figure 1). 

Regarding claim 30, Dimitrova discloses the video device is a computer systems 
(see col. 1, line 9). 

Regarding claim 31, Dimitrova discloses a device as discussed in the rejection of 
claim 30. Dimitrova further discloses the multimedia signal is a broadcast signal as 
discussed in the rejection of claim 13. Inherently, the device including a prefilter such as 
a tuner for filtering a multichannel input to provide the multimedia signal based on the at 
least one user preference. 

Regarding claim 32, Dimitrova discloses the "prefilter" filters the multichannel 
input based on a program guide (see col. 1 1 , lines 48-52). 

Regarding claim 33, Dimitrova discloses a video retrieval device wherein the 
combination of parsers and event model recognition devices generates identified 
keyframes for each segment (see col. 4, lines 26-28); and when a known keyframe can 
be specified as the retrieval criteria, all of the extracted keyframes are compared with 
the given keyframe. The comparison is performed using content-based image retrieval 
(see col. 15, lines 1-36). Inherently, the system includes a means for rendering one or 
more key frames associated with one or more segments of the plurality of segments, 
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and a means for selecting the selected segment based on the rendering of the one or 
more key frames. 

Regarding claim 34, Dimitrova discloses when a known keyframe can be 
specified as the retrieval criteria, all of the extracted keyframes are compared with the 
given keyframe. The comparison is performed using content-based image retrieval (see 
col. 15, lines 1-36), and order the selected set of multimedia signal clips according to 
the similarity and to display the results of the selected set (see col. 14, line 55 - col. 15, 
line 13). Inherently, the video retrieval device including a means for identifying one or 
more user references, and the means for rendering the one or more key frames 
includes a means for determining a comparison between a classification of each 
segment of the plurality of segments and the one or more user preferences and the 
rendering of the one or more key frames is dependent upon the comparison. 

Regarding claim 37, Dimitrova discloses a device as discussed in the rejection of 
claim 33. Inherently, the device including a means for rendering the selected segment 
on the display device 30. 

Regarding claim 39, Dimitrova discloses the label of the FA model is assigned by 
the user (see col. 11, lines 48-50), or the user inputs keyword or image (see col. 15, 
lines 1-36). Inherently, the means for selecting the selected segment includes at least 
one of a pointing device such as mouse or keyboard. 
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Claim Rejections - 35 USC § 103 



5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 11-12, 38, 40 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Dimitrova (US 6,363,380) as applied to claim 1 above, and in view of Dimitrova et 
al (US 6,137,544). 

Regarding claim 11, Dimitrova discloses a system as discussed in the rejection 
of claim 10. However, Dimitrova does not disclose encoding of each story segment of 
the plurality of story segments. 

Dimitrova et al. discloses encoding of each story segment of the plurality of story 
segment (see col. 3, lines 5-10). Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Dimitrova by 
utilizing encoding of each story segment of the plurality or story segments as taught by 
Dimitrova et al. in order to reduce the bandwidth. Inherently, the key frames are 
determined based upon a transform of an encoding of the story segment. 
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Regarding claim 12, Dimitrova et al. discloses the transform includes a discrete 
cosine transform, and the encoding is in an MPEG encoding (see col. 3, lines 5-34). 

Regarding claim 38, Dimitrova et al discloses the VCR retrieves the selected 
keyframes from the index memory of the source video, the retrieved keyframes are 
transferred to the host processor 210 which writes the processed keyframes to display 
memory and displays them in a user interface such as computer display, television 
screen. Furthermore, Dimitrova et al discloses the user can view any video source by 
clicking a keyframes on the index (see col. 13, lines 5-39). Inherently, the device 
including a rendering control for receiving render mode options and a means for 
rendering portions of each segment of the plurality of segments in dependence upon the 
render mode options. 

Regarding claim 40, Dimitrova et al discloses if user wish to view the source 
video at a particular keyframe, the user can select a keyframes from the index and the 
source tape could be automatically forwarded to a corresponding point on the source 
tape from where the keyframe was extracted and the source tape could be played. 
Inherently, the means for rendering the key frame of plurality of segments includes a 
multi-dimensional presentation of at least one of key frames. 
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7. Claims 18-22, 24-26 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Dimitrova (US 6,363,380) as applied to claim 17 above, and in view of Stern (US 
6,161,107). 

Regarding claim 18, Dimitrova discloses a system as discussed in the rejection 
of claim 17. However, Dimitrova fails to disclose the "filter" includes a sorter that 
associates a ranking to each story segment based on a correlation of the one or more 
preferences, and the one or more filtered story segments are identified based on the 
ranking associated with each story segment. 

Stern discloses a system for retrieving information component based on 
keyword, visual example and graphic attributes, wherein the relevance ranking or the 
retrieved information component is determined both by the number and density of 
required keywords which appear in the information component, if any, but also is 
preferably calculated according to the desired visual attributes and relationships to other 
information components. The system includes a mechanism for learning the 
preferences and profile of an individual user, which can then also be used to calculate 
the relevance ranking of the retrieved information (see col. 7, lines 64- col. 8, line 21). 
Therefore, it is obvious to one of ordinary skill in the art at the time the invention was 
made to modify Dimitrova to provide a "sorter" as taught by Stern in order to give a 
efficient result of the search to user. 
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Regarding claim 19, Stern discloses information components are displayed as 
separate icons on the screen (see col. 20, lines 20-21). 

Regarding claim 20, Stern discloses a profiler that produces the one or more 
preferences (see col. 8, lines 18-20 or col. 16, lines 61-67). 

Regarding claim 21, Stern discloses the one or more classification include 
keyword (see col. 7, lines 65-67). 

Regarding claim 22, Dimitrova discloses storage device 20 coupled to the display 
device 30 (see figure 1). Inherently, the display device 30 presents a selected story 
segment of the one or more filtered story segments based upon the one or more key 
frames that are presented on the display at a time when a user effects a selection. 

Regarding claim 24, Dimitrova discloses the system in the rejection of claim 17, 
further including a storage device 20 for storing plurality of story segments (see figure 
1). 

Regarding claim 25, Dimitrova discloses the storage device 20 is a VCR (see col. 
8, lines 50-51). 
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Regarding claim 26, Stern discloses displaying text that is associated with the 
filtered information components. 

8. Claim 23 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Dimitrova (US 6,363,380) in view of Stern (US 6,161,107) as applied to claim 22 above, 
and further in view of Gilbert (US 6,337,683). 

Regarding claim 23, Dimitrova in view of Stern discloses a system as discussed 
in the rejection of claim 22. However, neither Dimitrova nor Stern discloses the storage 
device presents a portion of each of the one or more filtered story segments 
sequentially. 

Gilbert discloses the viewer 22 displays in sequence a series in images, that is, a 
series of key frames. For each key frame displayed the viewer 22 determines an 
appropriate view window. The portion of the key frame which corresponds to the view 
window is then de-compressed and displayed (see col. 5, lines 1-8). Inherently, the 
storage device presents a portion of each of the one or more filtered story segments 
sequentially. Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify Dimitrova and Stern to provide a storage 
device presents a portion of each of the one or more filtered story segments 
sequentially as taught by Gilbert in order to allow a user to access a video program in 
order. 
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9. Claims 35-36 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Dimitrova (US 6,363,380as applied to claim 34 above, and in view of Gilbert (US 
6,337,683). 

Regarding claim 35, Dimitrova discloses a device as discussed in the rejection of 
claim 34. Dimitrova further discloses display device 30 for display one or more key 
frames. However, Dimitrova fails to disclose the one or more key frames associated 
with each of the one or more panes segments are displayed sequentially in the one or 
more panes. 

Gilbert discloses the viewer 22 displays in sequence a series in images, that is, a 
series of key frames. For each key frame displayed the viewer 22 determines an 
appropriate view window. The portion of the key frame which corresponds to the view 
window is then de-compressed and displayed (see col. 5, lines 1-8). Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to modify Dimitrova to have key frames displayed sequentially in the viewer as taught by 
Gilbert in order to allow a user to understand the video program easily. 

Regarding claim 36, Gilbert discloses the portion of the key frame which 
corresponds to the view window is then de-compressed and displayed (see col. 5, lines 
1-8). 
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Conclusion 



10. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Yeo et al. (US 5,821 ,945) discloses method and apparatus for video browsing 
based on content and structure. 

Bokser et al. (US 5,862,259) discloses pattern recognition employing arbitrary 
segmentation and compound probabilistic evaluation. 

Elenbaas etal. (US 6,119,123) discloses method and apparatus for optimizing 
keyframe and blob retrieval and storage. 

Hullinger et al. (US 6,295,092) discloses a system that automatically captures 
one or more local news program broadcast and separates the broadcast into the 
individual news stories or segments. 

Zhang et al. (US 5,635,982) discloses an automatic video content parser for 
parsing video shots. 
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Ceccarelli (US 6,222,532) discloses method and device for navigating through 
video matter by means of displaying a plurality of keyframes in parallel. 

Lyberg (US 5,752,227) discloses method and arrangement for speech to text 
conversion. 

Niblack (US 6,182,069) discloses video query system and method. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Son P Huynh whose telephone number is 703-305- 
1889. The examiner can normally be reached on 8:00-5:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Faile can be reached on 703-305-4380. The fax phone numbers for 
the organization where this application or proceeding is assigned are 703-872-9314 for 
regular communications and 703-872-9314 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the customer service office whose telephone number 
is 703-306-0377. 




Son P. Huynh 
May 20, 2002 
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SUPERVISORY PATENT EXAMINER 
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